Papers with cross-entropy loss function
Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss (N18-1)
Copied to clipboard
| Challenge: | Existing methods for fine-grained type classification rely on distant supervision and are susceptible to noisy labels that can be out-of-context or overly-specific. |
| Approach: | They propose a neural network model that uses cross-entropy loss function to handle out-of-context labels and hierarchical loss normalization to cope with overly-specific ones. |
| Outcome: | The proposed model outperforms the state-of-the-art on established benchmarks for the task. |
From Narrow Unlearning to Emergent Misalignment in LLMs (2026.acl-short)
Copied to clipboard
Erum Mushtaq, Anil Ramakrishna, Satyapriya Krishna, Sattvik Sahai, Prasoon Goyal, Kai-Wei Chang, Tao Zhang, Rahul Gupta
| Challenge: | Recent work shows that fine-tuning on insecure code data can trigger an emergent misalignment (EMA) phenomenon . |
| Approach: | They extend their study by demonstrating that EMA can arise from narrow refusal unlearning . they perform refusal unLearning on Cybersecurity and Safety concept and evaluate EMA . |
| Outcome: | The proposed model can generate malicious responses even to unrelated prompts . the proposed model is able to restore alignment across the affected domains while having lower refusal rates. |
Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to token-level adaptive training only use static word frequency information without considering the source language. |
| Approach: | They propose a bilingual mutual information based adaptive objective that assigns weights to target tokens with higher BMI . they propose to use this approach to improve token-level adaptive training . |
| Outcome: | The proposed method improves token-level adaptive training on two languages. |
Improving Dialogue State Tracking with Turn-based Loss Function and Sequential Data Augmentation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rely on a traditional cross-entropy loss function during training, which may not be optimal for improving the joint goal accuracy. |
| Approach: | They propose a Turn-based Loss Function that penalises the model if it inaccurately predicts a slot value at the early turns more so than in later turns to improve joint goal accuracy. |
| Outcome: | The proposed techniques improve the state-of-the-art model by approximately 7-8% relative reduction in error and achieve a new state- of-the art joint goal accuracy with 59.50 and 54.90 on MultiWOZ2.1 and MultiWOz2.2, respectively. |
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing diffusion models have limitations in modeling discrete data, e.g., languages . we present a novel diffusion model for language modeling inspired by linguistic features in languages based on iterative denoising . |
| Approach: | They propose a method that iteratively denoises and adds corruptions to the textual data through soft-masking to better noise it. |
| Outcome: | The proposed model achieves better generation quality and lower training cost than current models with better performance. |
Content Explorer: Recommending Novel Entities for a Document Writer (D18-1)
Copied to clipboard
| Challenge: | Existing tools for exploratory search can be useful for document writers but they are not always effective for identifying topics for further research. |
| Approach: | They propose a supervised learning problem for recommending topics to a writer . they propose entropy loss function modification to improve the results . |
| Outcome: | The proposed model improves on a large dataset and can be used to identify topics for further research. |
A Simple Log-based Loss Function for Ordinal Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for ordinal text classification do not incorporate ordinal character into their feedback. |
| Approach: | They propose a new ordinal log-loss loss function that incorporates ordinal character into its feedback. |
| Outcome: | The proposed loss function outperforms state-of-the-art methods on four benchmark text classification datasets. |
Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classification (D19-1)
Copied to clipboard
| Challenge: | Existing models for multi-label classification ignore complexity and dependencies among labels . Experimental results show that our method can obtain more accurate multi-lab classification results. |
| Approach: | They propose a meta-learning method to capture complex label dependencies . they use a Meta-learner to jointly learn the training policies and prediction policies for different labels. |
| Outcome: | The proposed method can capture complex label dependencies on fine-grained entity typing and text classification tasks. |